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Block artifacts detection 



The invention relates to a block artifacts detection device for detecting block 
artifacts in a video signal. 

The invention further relates to an image processing apparatus comprising: 
receiving means for receiving a video signal corresponding to a sequence of 

5 input images; 

such a block artifacts detection device; and 

an image processing unit for calculating a sequence of output images on basis 
of the sequence of input images, the image processing unit being controlled by the block 
artifacts detection device. 
10 The invention further relates to a method of detecting block artifacts in a video 

signal. 

The invention further relates to a computer program product to be loaded by a 
computer arrangement, comprising instructions to detect block artifacts in a video signal, the 
computer arrangement comprising processing means and a memory. 

15 

The appearance of so-called blocking artifacts or block artifacts in video 
frames, caused by digital block-based encoding schemes, e.g. MPEG2, or H.264, has become 
an increasing problem in the field of video processing. Especially on high-quality display 
20 devices, the perceived image quality can be degraded substantially by these artifacts. To 
make matters worse, image enhancement units, not only enhance edges of the source 
material, but can also boost these block-edge artifacts, deteriorating the image quality even 
further. 

The block artifacts are introduced in the transmission chain before reception 
25 by a consumer device, e.g. a television. The appearance of block artifacts is caused by an 
imperfect and lossy compression scheme, which processes individual blocks of pixels 
independently. These digital coding artifacts can appear, for example, in a lossy compression 
before satellite transmission, after which the video signal may be further broadcasted by 
analog means. In such a typical case, information on the position and size of the blocks or 
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any other parameter from the digital compression is not directly available in the analog video 
signal. In order to assess the presence and visibility of block artifacts, a device and a method 
is required to extract this information from the video signal: to locate the artifacts and 
measure their visibility. 
5 Block artifact indicators represent this type of information. Block artifact 

indicators can be applied to control further image processing. For example to control (or turn 
off) a sharpening unit in the case of encountering a video signal with relatively many block 
artifacts. Alternatively processing, e.g. smoothing can be applied to reduce these block 
artifacts. 

10 An embodiment of the method of the kind described in the opening paragraph 

is known from WO 01/20912. The known method comprises a step of filtering the input 
signal with a gradient filter to provide a filtered signal and a step of calculating a block level 
metric, i.e. a block artifact indicator, for processing the filtered signal to identify and count 
blocking artifacts as a function of their position in a grid. The known method works 

15 appropriately for a limited set of predetermined block grid sizes. Unfortunately, the actual 
spatial size of the block artifacts in the received video signal often differs from the sizes of 
the limited set of predetermined block grid sizes because of spatial scaling of the image data 
being represented by the video signal, somewhere in the chain from transmission to 
reception. 

20 



It is an object of the invention to provide a block artifacts detection device of 
the kind described in the opening paragraph which is relatively robust. 

This object of the invention is achieved in that the block artifacts detection 
25 device comprises : 

computing means for computing a gradient signal on basis of the video signal; 

establishing means for establishing a list of samples corresponding to 
respective local maximum values of the gradient signal; 

histogram determining means for determining a histogram of inter-sample 
30 distances, a first one of the inter-sample distances corresponding to a first distance between a 
first one of the samples and a second one of the samples succeeding the first one of the 
samples, and a second one of the inter-sample distances corresponding to a second distance 
between the first one of the samples and a third one of the samples succeeding the second one 
of the samples; and 
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analyzing means for analyzing the histogram of inter-sample distances and for 
producing a block artifact indicator on basis of the histogram. 

An important aspect of the invention is that a histogram of inter-sample distances is made on 
basis of all distances between samples of the group of samples corresponding to respective 
5 local maximum values of the gradient signal, within a sliding window. That means all 
distance between samples within a moving aperture located on a portion of the list of 
samples. Hence, not only the distances between adjacent samples are taken into account but 
all mutual distances between samples in a spatial neighborhood. Besides that, there is no a 
priori distance applied, i.e. a predetermined number of pixels distance considered, while 

10 establishing the histogram. That means that not only distances between samples of e.g. 8 
pixels are counted but all distances between samples within the extent of the aperture, 
expressed in an integer number of pixels. By means of appropriate analyzing the histogram 
comprising the information of all distances, the block artifact indicator is provided. The 
analyzing comprises the selection of a dominant bin, coiTesponding to a particular inter- 

15 sample distance, from the histogram and optionally combining the value of that bin with 
values of neighboring bins. 

In an embodiment of the block artifacts detection device according to the 
invention, the block artifact indicator corresponds with a spatial size of the block artifacts, the 
block artifact indicator being related to a particular inter-sample distance. The spatial size of 

20 the block artifacts can relatively easy be determined by directly applying the selected bin, i.e. 
inter-sample distance. Preferably the block artifact indicator is computed on basis of the 
value of that bin with values of neighboring bins. This allows to compute the spatial size of 
the block artifacts with sub-pixel accuracy. Because of spatial scaling of the video data, the 
block size might e.g. be 10 2/3 pixel. 

25 In an embodiment of the block artifacts detection device according to the 

invention, the block artifact indicator corresponds with a measure of visibility of the block 
artifacts, the block artifact indicator being related to a frequency of occurrence of a particular 
inter-sample distance. It has been proven that the frequency of occurrence or the relative 
frequency of occurrence of the particular inter-sample distance is a good indicator for the 

30 visibility of the block artifacts. Optionally, also the values of two neighboring bins of the 
selected bin are taken into for the computation of the block artifact indicator corresponding 
with a measure of visibility of the block artifacts. 

In an embodiment of the block artifacts detection device according to the 
invention, the histogram of inter-sample distances is a weighted histogram. That means that 
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the distances are not just counted but that the contribution of each of the distances to the 
histogram is based on a respective weight For example, the weighting of the first distance is 
based on the local maximum value of the first one of the samples. Optionally the weighting 
of the first distance is also based on the local maximum value of the second one of the 
5 samples. Preferably, the weighting of the first distance is based on a portion of the gradient 
signal comprising a sub-portion corresponding to the first one of the samples. In other words, 
also values of the gradient signal around the local maximum value are taken into account for 
the weighting. An advantage of applying a weighted histogram is that the noise robustness is 
further increased. 

10 In an embodiment of the block artifacts detection device according to the 

invention, the gradient signal is computed on basis of a first intermediate signal being 
computed by summation of respective pixel values of a number of video lines of the video 
signal. This summation is a kind of low-pass filtering. An advantage of this embodiment is 
that the noise robustness is further increased. 

15 In an embodiment of the block artifacts detection device according to the 

invention, the gradient signal is computed by high-pass filtering of a first intermediate signal 
which is based on computing absolute differences between subsequent pixel values of the 
video signal. This high-pass filtering enables to apply a robust thresholding in order to create 
the list of relevant local maximum values. That means that non-relevant local maximum 

20 values, having a value below a predetermined threshold are disregarded. 

It is a further object of the invention to provide an image processing apparatus, 
of the kind described in the opening paragraph, comprising a block artifacts detection device 
which is relatively robust. 

This object of the invention is achieved in that the block artifacts detection 

25 device comprises: 

computing means for computing a gradient signal on basis of the video signal; 
establishing means for establishing a list of samples corresponding to 
respective local maximum values of the gradient signal; 

histogram determining means for determining a histogram of inter-sample 

30 distances, a first one of the inter-sample distances corresponding to a first distance between a 
first one of the samples and a second one of the samples succeeding the first one of the 
samples, and a second one of the inter-sample distances corresponding to a second distance 
between the first one of the samples and a third one of the samples succeeding the second one 
of the samples; and 
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analyzing means for analyzing the histogram of inter-sample distances and for 
producing a block artifact indicator on basis of the histogram. 

The image processing apparatus may comprise additional components, e.g. a display device 
for displaying the output images. The image processing unit might support one or more of the 
5 following types of image processing: 

Video compression, i.e. encoding or decoding, e.g. according to the MPEG 

standard. 

De-interlacing: Interlacing is the common video broadcast procedure for 
transmitting the odd or even numbered image lines alternately. De-interlacing attempts to 
10 restore the full vertical resolution, i.e. make odd and even lines available simultaneously for 
each image; 

Image rate conversion: From a series of original input images a larger series of 
output images is calculated. Output images are temporally located between two original input 
images; and 

15 - Temporal noise reduction. This can also involve spatial processing, resulting 

in spatial-temporal noise reduction. 

The image processing apparatus might e.g. be a TV, a set top box, a VCR (Video Cassette 
Recorder) player, a satellite tuner, a DVD (Digital Versatile Disk) player or recorder. 

It is a further object of the invention to provide a method, of the kind 
20 described in the opening paragraph which is relatively robust. 

This object of the invention is achieved in that the method comprises: 

computing a gradient signal on basis of the video signal; 

establishing a list of samples corresponding to respective local maximum 
values of the gradient signal; 
25 - determining a histogram of inter-sample distances, a first one of the inter- 

sample distances corresponding to a first distance between a first one of the samples and a 
second one of the samples succeeding the first one of the samples, and a second one of the 
inter-sample distances corresponding to a second distance between the first one of the 
samples and a third one of the samples succeeding the second one of the samples; and 
30 - analyzing the histogram of inter-sample distances and producing a block 

artifact indicator on basis of the histogram. 

It is a further object of the invention to provide a computer program product, 
of the kind described in the opening paragraph which is relatively robust 
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This object of the invention is achieved in that the computer program product, 
after being loaded, providing said processing means with the capability to carry out: 

computing a gradient signal on basis of the video signal; 

establishing a list of samples corresponding to respective local maximum 
5 values of the gradient signal; 

determining a histogram of inter-sample distances, a first one of the inter- 
sample distances corresponding to a first distance between a first one of the samples and a 
second one of the samples succeeding the first one of the samples, and a second one of the 
inter-sample distances corresponding to a second distance between the first one of the 
10 samples and a third one of the samples succeeding the second one of the samples; and 

analyzing the histogram of inter-sample distances and producing a block 
artifact indicator on basis of the histogram. 

Modifications of the block artifacts detection device and variations thereof 
may correspond to modifications and variations thereof of the image processing apparatus, 
15 the method and the computer program product, being described. 

These and other aspects of the block artifacts detection device, of the image 
processing apparatus, of the method and of the computer program product, according to the 
20 invention will become apparent from and will be elucidated with respect to the 
implementations and embodiments described hereinafter and with reference to the 
accompanying drawings, wherein: 

Fig. 1 schematically shows an embodiment of the block artifacts detection 

Fig. 2 shows an input image; 

Fig. 3 shows an example of a gradient signal S , based on the image of Fig. 2; 
Fig. 4 shows the detrended gradient signal s based on the gradient signal S 

Fig. 5 shows a portion of the detrended gradient signal s as shown in Fig. 4; 

Fig. 6 shows a weighted inter-peak distance histogram H* ; 
Fig. 7 shows an example of g(d) ; and 

Fig. 8 schematically shows an embodiment of the image processing apparatus 
400 according to the invention. 



device; 

25 

of Fig. 3; 

30 
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Same reference numerals are used to denote similar parts throughout the Figures. 

Fig. 1 schematically shows an embodiment of the block artifacts detection 
5 device 100 according to the invention. The block artifacts detection device 100 is provided 
with a video signal at the input connector 110 and is arranged to provide a control signal 
representing the detected block artifacts, at its output connector 1 12. The control signal is 
related to the detected block artifacts. The block artifacts detection device 100 comprises: 

a computing unit 102 for computing a gradient signal S on basis of the video 

10 signal; 

a maximum detecting unit 104 for establishing a list of samples corresponding 
to respective local maximum values 402-408 of the gradient signal; 

a histogram determining unit 106 for determining a histogram of inter-sample 
distances, a first one of the inter-sample distances corresponding to a first distance between a 
15 first one of the samples and a second one of the samples succeeding the first one of the 
samples; and 

an analyzing unit 108 for analyzing the histogram of inter-sample distances 
and for producing a block artifact indicator on basis of the histogram. 

Preferably, the histogram determining unit 106 is arranged to create a 
20 weighted histogram as described in connection with Fig. 5. This measure can be seen as a 
separate invention. Preferably all inter-sample distances within a sliding window are 
determined. The working of the block artifacts detection device 100 is described in 
connection with the Figs. 2-7. 

The computing unit 102, the maximum detecting unit 104, the histogram 
25 determining unit 106 and the analyzing unit 108 may be implemented using one processor. 
Normally, these functions are performed under control of a software program product. 
During execution, normally the software program product is loaded into a memory, like a 
RAM, and executed from there. The program may be loaded from a background memory, 
like a ROM, hard disk, or magnetically and/or optical storage, or may be loaded via a 
30 network like Internet. Optionally an application specific integrated circuit provides the 
disclosed functionality. 

Fig. 2 shows an input image, in particular a luminance field being grabbed 
from the National Geographic Channel. Note the appearance of regular blocks as a result of a 
high compression ratio. In this example, block artifacts appear with a period of 10 2/3 pixel. 



WO 2005/015915 



PCT/IB2004/051322 



8 

The image format is Standard Definition (SD): 288 lines of each 720 pixels. Below will be 
explained how the block artifact indicators are computed. This is based on detection of 
vertical edges. An analogous approach then holds for horizontal edges. 

Define an M xN image I with elements I tJ where i and j are integer grid 

5 positions. A first step is the computation of a gradient signal. This comprises the computation 

of the absolute horizontal gradient vector D of which the elements are specified by 
Equation 1: 

D o =k»v- 7 *l (1) 
Next, all relevant transitions are counted or, in other words, any pixels that are 
10 located on an edge. Thus, the occurrences are counted, along y-direction, that the absolute 
gradient exceeds a first predetermined threshold 0 . A typical value of 0 equals 2 with the 
values of I ranging form 0-255. This yields a vector S with elements Sj : 

'.-■spin-*) 

where j = 1,2,..., N and T(x) is a step-function as specified in Equation 3: 

is nx)=\ l x>0 

An example of S , based on the image of Fig. 2 is shown in Fig. 3. 

In the case that image / does contain notable MPEG block artifacts, it is 

— * 

expected that S is a signal that contains peaks, i.e. local maximum values, at regular 
intervals. The next step is to detect the repetition period of these peaks. It was found that 

20 Equation 2 provides a multi-peak signal that is more robust to the influence of original image 
edges, i.e. not the MPEG block artifacts, than just a row-sum of absolute gradient D . This 
can be understood from the notion that a large gradient is simply counted, just as a relatively 
small gradient. This reduces the relative contribution of a relatively large gradient to the 
average S . Typical MPEG block artifacts are not expected to create very large gradients, but 

25 are expected to be large enough to be clearly visible. The goal is therefore more to find how 
often an edge is found on a vertical image column, than to find the average edge size. In the 
latter case, large edges in the source material could result in dominant peaks in S , rather than 
the more moderate MPEG block edges. 



(2) 



(3) 
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Any significant peak in S is considered to be a the result of a suspected block 

edge, i.e. block artifact. In order to find suspected block edges, a detection of peaks in S is 
required. Before determining which peak exceeds a second predetermined threshold a , and 

can be considered as a relevant peak, the low frequent trend of S is subtracted from S . This 
5 is effectively a high-pass filtering and is achieved by subtracting from each value Sj the 

average of its direct 2n + 1 size neighborhood, j-n 9 j-n + 1,..., j + n . A typical value of 
n = 4. The detrended edge count Sj is specified in Equation 4: 

Fig. 4 shows the detrended edge count s based on the edge count signal S of 
10 Fig. 3. The dotted line 400 corresponds with the second predetermined threshold a . The dots 
402-408 indicate detected peaks above the second predetermined threshold a . Detrending S 
effectively comes down to normalizing each Sj with respect to its direct neighbors or, in 
other words, comparing the amount of edge found at j with the amount of edge found next 
to j . This makes sense as one considers that in detailed regions, with detailed textures, on 
15 average many edges will be detected. Hence, an edge is taken into account if it is not only 
high in absolute sense, but also in a relative sense. 

A next step is detection of relevant peaks, i.e. local maximum values. Suppose 

that there are portions of s that exceed the second predetermined threshold a . That 
means that the list of samples corresponding to respective local maximum values of the 
20 gradient signal comprises samples. The locations of the begin and end of the k-th peak 
504 are defined such that: 

if m k £j£ n k then Sj^a, k = 1,2,.. JV^ (5) 
The location p k of the k-th peak 504 is the index / for which a local maximum value of 
s occurs: 

25 m k £p k £n k9 s pt *s pk±l (6) 

This peak detection is illustrated in Fig. 5 for a portion of the detrended edge count s as 
shown in Fig. 4. m k 502 is at the first pixel above the second predetermined threshold a as 



WO 2005/015915 



PCT/IB2004/051322 



10 



10 

illustrated by means of the dotted line 400 and n k 506 is at the last pixel above the second 
predetermined threshold a . The volume V k of the k-th peak is then defined as: 

F * = (7) 

Preferably, this volume V k is used as a weight in the inter-peak histogram, i.e. histogram of 
inter-sample distances. 

In order to determine whether the detrended edge count s comprises repetitive 
peaks, i.e. relevant inter-sample distances, one could start the analysis by computing a 
histogram of the distances between peaks within a limited vicinity, i.e. window or aperture. 
The limit of this neighborhood N hist is determined by the maximum expected block size 
taking into account a scaling factor. A typical value N hist = 38 pixels. The inter-peak 

distance histogram H could be computed by means of the following piece of C-code: 
for (k=l; k <Nedge; k-H-) { 

while ( ((d = p[k+i] - p[k]) < Nhist) && (k+i <= Nedge) ) { 
15 H[d] 4= 1; /* add 1 to histogram */ 

i4=l; 

} 

} 

In the computation above, i/does not explicitly take any edge visibility into 
20 account: it merely counts the inter-peak distances. Thus it counts repetitive appearing edges, 
but regardless of the extent of the block edges. Preferably a weighted inter-peak distance 
histogram H w is computed. Fig. 6 shows the weighted inter-peak distance histogram H w . 
Note that the base period, i.e. block artifact grid size is non-integer, 10 2/3. This can be 
derived from the fact that the histogram volume is not concentrated in single bins but in pairs 
25 of bins, e.g. bin d = 10 and d = 1 1, and bin d = 21 and d = 22 . In order to take the edge 
visibility into account in the histogram H w as a weight, the volume measures V k of the 
edges, as defined in Equation 7 are used as a weight. In the current method, the distance 
between two samples is weighted with the volumes of both samples. The weighted inter-peak 
distance histogram H w could be computed by means of the following piece of C-code: 
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for (k=l ; k < Nedge; k++) { 
i=l; 

while ( ((d = p[k+i] - p[k]) < Nhist) && (k+i <= Nedge) ) { 

H[d] -H= (V[k] + V[k+i])/2; /* add volumes as weight to histogram */ 
i4= 1; 

> 

} 

where V[k] corresponds to V k as defined in Equation 7. 

Next, in order to determine repetitive peaks, the weighted inter-peak distance 

histogram H„ is sampled at integer multiples, k of each possible base period d , thus at 
kd: 



(8) 



If the underlying base period were d , then the histogram would peak at integer multiples, 
k - d . Therefore, g(d) would attain its maximum for the actual, underlying base period d . 

15 The location of a clear, first maximum of g(d) is then taken as an indicator of the underlying 
base period It was found that finding the first peak in g(d) lead to a more stable estimate of 
the underlying base period d , than detection of the first prominent histogram bin H d . The 
reason is that, in practical cases, it appears that the histogram at \xd is not always that 
prominent. For example, suppose that the base period is, in fact, d = 8 , it appears that, in 

20 some cases H 9 is not that prominent, whereas H l6 and clearly are. Such a case does not 
necessarily pose a problem, if a first peak is searched in g(d) rather than H d . See Fig. 7 for 
an example of g(d) . In this Fig. 7 the inter-peak distance d is sampled at 0.2 intervals. 

An aspect of the invention is that the block artifacts detection device is 
arranged to produce a control signal, which indicates the visibility of blocking artifacts, 

25 without further need of information on coding parameters used earlier in the processing 
chain. Processing in television sets and video recording devices e.g. DVEH-RW and Hard 
Disk Recorders can be adapted to these cases where a significant amount of blocking edges 
are detected. The first block artifact indicator corresponds with a measure of visibility of the 
block artifacts. The first block artifact indicator is related to a frequency of occurrence of a 

30 particular inter-sample distance and preferably equal to the mean of a number of values, 
including the relative frequency of occurrence of the particular inter-sample distance. 
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Another aspect of this invention is that the block artifacts detection device is 
arranged to provide a control signal, which indicates one or more dimensions of the blocking 
grid without the need for information that is not explicitly present in the analog video signal. 
This information can also be used by the processing, as it indicates whether a scaling 
5 operation has been applied, assiiming that the applied coding scheme uses blocks with a 

width and height of eight pixels. The second block artifact indicator d , coiresponding with a 
spatial size of the block artifacts, is related to a particular inter-sample distance. However, the 
second block artifact indicator d is not necessarily equal to a particular inter-sample distance, 
which has an integer value. 

10 Fi S- 8 schematically shows an embodiment of the image processing apparatus 

400 according to the invention, comprising: 

receiving means 802 for receiving a video signal corresponding to a sequence 
of input images; 

a block artifacts detection device 100 for detecting block artifacts in the video 
1 5 signal, as described in connection with Fig 1 ; 

an image processing unit 804 for calculating a sequence of output images on 
basis of the sequence of input images, the image processing unit being controlled by the 
block artifacts detection device; and 

a display device 806 for displaying the.output images of the image processing 

20 unit 804. 

The signal may be a broadcast signal received via an antenna or cable but may also be a 
signal from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk 
(DVD). The signal is provided at the input connector 810. The image processing apparatus 
800 might e.g. be a TV. Alternatively the image processing apparatus 800 does not comprise 

25 the optional display device but provides the output images to an apparatus that does comprise 
a display device 806. Then the image processing apparatus 800 might be e.g. a set top box, a 
satellite-tuner, a VCR player, a DVD player or recorder. Optionally the image processing 
apparatus 800 comprises storage means, like a hard-disk or means for storage on removable 
media, e.g. optical disks. The image processing apparatus 800 might also be a system being 

30 applied by a film-studio or broadcaster. 

It should be noted that the above-mentioned embodiments illustrate rather than 
limit the invention and that those skilled in the art will be able to design alternative 
embodiments without departing from the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not be constructed as limiting the claim. 
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The word 'comprising* does not exclude the presence of elements or steps not listed in a 
claim. The word "a" or "an" preceding an element does not exclude the presence of a 
plurality of such elements. The invention can be implemented by means of hardware 
comprising several distinct elements and by means of a suitable programmed computer. In 
5 the unit claims enumerating several means, several of these means can be embodied by one 
and the same item of hardware. 



